Skip to content

cache data columns with missing parent - #157

Merged
vladimir-ea merged 17 commits into
mainfrom
ve/column_cache
Aug 6, 2026
Merged

cache data columns with missing parent#157
vladimir-ea merged 17 commits into
mainfrom
ve/column_cache

Conversation

@vladimir-ea

@vladimir-ea vladimir-ea commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

A few different fixes related to syncing / following:

  • increase frequency of TCache producer head publication so that idle consumers reset before the buffer is empty
  • refresh target on head sync if wall slot has moved during sync: previously we would switch to following even if far behind all slot
  • handle tiny gossip messages (sent by 1.3.0 supporting clients)
  • cache data columns for unknown parent blocks rather than discarding - validate when missing parent block arrives

Comment thread crates/storage/src/tile.rs Outdated
Comment thread crates/storage/src/tile.rs
"sidecar parent_root not yet validated — ignoring (not penalized)"
);
return ColumnOutcome::Skip;
return ColumnOutcome::AwaitParent { parent_root: *parent_root };

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we do cache these now, maybe we should do signature verification first? to filter out invalid data

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we do signature verification once per block and then its the last check after everything else - so these cached entries would fail validation when the parent block is available - not sure it is worth verifying up front?

Comment thread crates/control/src/sync_engine/select.rs Outdated
Comment thread crates/storage/src/tile.rs Outdated
Comment on lines +654 to +656
self.validate_gloas_column(stream_id, buf, gossip_subnet)
} else {
self.validate_fulu_column(stream_id, buf, gossip_subnet, None)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these have been run before caching already I think?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parent check at least needs to be re-run - will need to re-jig this

Comment thread crates/storage/src/tile.rs Outdated
Comment thread crates/network/src/p2p/streams/gossip_in.rs
@vladimir-ea
vladimir-ea merged commit 7c39032 into main Aug 6, 2026
1 of 3 checks passed
@vladimir-ea
vladimir-ea deleted the ve/column_cache branch August 6, 2026 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants